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DETAILED ACTION 

This is a Final Office Action responsive to amendments and remarks filed 
10/16/2008. Claims 1-9; 11; 13-27; 29-39; 41-47; 49-56; 58-64 and 66-69 are 
currently pending in this application. Claim 12 was previously cancelled, claims 
1 0; 28; 40; 48; 57 and 65 are currently cancelled. Independent claims 1 ; 21 ; 34; 
44; 51; 61; 68 and 69 have been amended; Effective filing date is 03/30/2004, 
(Assignee: Ricoh). 

Claims Rejection - 35 U.S.C. 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for 
all obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or 
described as set forth in section 102 of this title, if the differences between the subject 
matter sought to be patented and the prior art are such that the subject matter as a 
whole would have been obvious at the time the invention was made to a person having 
ordinary skill in the art to which said subject matter pertains. Patentability shall not be 
negatived by the manner in which the invention was made. 

Claims 1-9; 11; 13-14; 17-27; 29-30; 33-39; 41, 43-47; 49, 51-56; 58, 60- 
64 and 66-69 are rejected under 35 U.S.C. 103(a) as being as being 
unpatentable over Chiu et al US007051271 B1 filed 10/03/2000 (hereinafter 
Chiu), in view of Hull [Document image similarity and equivalence protection] 
Published 1 1/29/1 19 by " IJDARUS" Pages 37-42 (hereinafter Hull), and further 
view of Bozdaqi et al -US006647535B1 1 filed 08/18/1999 (hereinafter Bozdagi), 
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Regarding independent claims 1, 

Chiu teaches: 

a method in a computer system for creating a composite 
electronic representation including presentation material 
information, the computer system thus creating a composite 
electronic representation of the document 

(See Chiu at the Abstract, teaches a computer system for creating a composite 
electronic representation including presentation material information (i.e. 
annotations made on the paper documents during the meeting can be extracted 
and used as indexes to the video). This interpretation is supported by Applicant's 
specification, which is stated "creating a composite electronic representation 
comprising the determined information" at Pages 3 Para 15. ) 

the method comprising: scanning a paper document to 
generate an electronic presentation of the document with 
presentation material. 

(See the Abstract and Col. 2, Lines 15-25^Chiu discloses this limitation in that a 
computer system for creating a composite electronic representation including 
presentation material information (i.e. annotations made on the paper documents 
during the meeting can be extracted and used as indexes to the video.) in 
addition Chiu further discloses the a scanned document is linking to a segment of 
a video wherein the scanned document is linked to a first video frame in the 
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plurality of video frames responsive to the comparison step includes an identifier 
for comparing to the plurality of video frame identifiers; 

This interpretation is supported by Applicant's Specification, which states 
"key frame images obtained from video information captured during the 
presentation" at Pages 21 Para 57. ) 

extracting a visual feature from the electronic presentation of 
the document, the visual feature corresponding to a portion of the 
presentation material; 

(See Chiu at the Abstract, teaches annotations made on the paper documents 
during the meeting can be extracted and used as indexes to the video. An 
orthonormal transform, such as a Digital Cosine Transform (DCT) is used to 
compare scanned documents to video frames.) 

accessing recorded information including at least one of audio 
and visual information recorded during a presentation of the 
presentation material; 

(See Chiu at Col. 2, Lines 15-25, discloses a method for linking a scanned 
document to a segment of a video is provided. Whereby the scanned document 
identifier is then compared to the plurality of video frame identifiers; the scanned 
document is linked to a first video frame in the plurality of video frames 
responsive to the comparison step. 
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Also see Chiu at Fig. 3 and Col. 4, Line 1-15, discloses browser interface 
300 for accessing a segment of video file 200 which references scanned 
document 101a.) 




5L : d sM FIG. 3 

storing the composite electronic representation for access by 
the user or another user accessing the composite electronic 
document. 

(See Chui at Fig. 5 and Col. 8 Line 5 discloses memory item 806 and persistent 
storage 808 stores a scanned digital document 101a and video file 200.) 

Also see Chiu at Col. 2, Lines 15-25, discloses a method for linking a 
scanned document to a segment of a video is provided. Whereby the scanned 
document identifier is then compared to the plurality of video frame identifiers; 
the scanned document is linked to a first video frame in the plurality of video 
frames responsive to the comparison step. 
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Also see Chiu at Fig. 3 and Col. 4, Line 1-15, discloses browser interface 
300 for accessing a segment of video file 200 which references scanned 
document 101a. 

Also see Chiu at Fig. 5 and Col. 8 Line 5 discloses memory item 806 and 
persistent storage 808 stores a scanned digital document 101a and video file 
200.) 
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metadata including the matching information, 

(See Chiu at Col. 2, Lines 15-25, discloses a method for linking a scanned 
document to a segment of a video is provided. Whereby the scanned document 
identifier is then compared to the plurality of video frame identifiers; the scanned 
document is linked to a first video frame in the plurality of video frames 
responsive to the comparison step. Also Chiu further illustrated at FIG. 2 browser 
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interface 100a having window 105 displays the relative time in which video frame 
200a is displayed during the course of event 50. 

This interpretation is supported by Applicant's Specification, which states 
"metadata may indicate how long the slide was discussed" at Pages 14 Para 
[0066] Lines 11-12. ) 

^ 103 ^ 1QOa 
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FIG. 2 



In addition Chui does not expressly teach, but Hull teaches: 

comparing the visual feature to the recorded information to 
determine a portion of the recorded information that matches the 
visual feature, whereby at least a portion of the recorded information 
matches a feature portion of the presentation material, and 
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determining matching information for each matching portion of the 
recorded information and feature portion using a matching algorithm 
configured to map the visual feature to a portion of any of a plurality 
of recorded information that matches the feature portion; and of the 
recorded information determined to match the visual feature, 
(See Fig. 2 and at Pages 308-31 2-> Hull discloses this limitation in that features 
extracted from the CCITT fax-compressed representations of two images are 
compared to determine their visual similarity and whether they are equivalent. As 
illustrated in Fig. 2 hierarchical algorithm is presented for determining the 
similarity and equivalence of document images which are extracted from the 
CCITT fax-compressed representations of two images are compared to 
determine their visual similarity and whether they are equivalent. The 
equivalence of two documents is determined by applying the Hausdorff distance 
to the two-dimensional arrangement of pass codes in small patches of each 
image [matching portion of recorded images]. In this case the fax-compress 
images are compared to the compress image form the data base to determine 
their visual similarity and whether they are equivalent. Furthermore, Hull further 
disclosed the similarity detection algorithm include incorporation of structural 
information about the segmentation of a document into zones such as text, 
graphics, photographs, etc. Such information may improve performance. 
However, the run time overhead of such a method should be considered. The 
experiments with the equivalence detection method assume corresponding 
image patches can be located in two equivalent images. An automatic technique 
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for calculating this from compressed data should be developed [at Page 312 Top 
right column of Hull.) 
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Figure 2. Document image similarity and equivalence detection algorithm, 



Accordingly, It would have been obvious to one of ordinary skill in the art 
at the time of the invention to modify Chui method of linking a video to a scan 
document to include a means of comparing the visual feature to the recorded 
information to determine a portion of the recorded information that matches the 
visual feature, whereby at least a portion of the recorded information matches a 
feature portion of the presentation material, and determining matching 
information for each matching portion of the recorded information and feature 
portion using a matching algorithm configured to map the visual feature to a 
portion of any of a plurality of recorded information that matches the feature 
portion as taught by Hull. One of ordinary skill in the art would have been 
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motivated to modify this combination because Hull and Chui are from the same 
field of endeavor of electronic presentation material associated with the scan 
document, and provides a predictable result of generating a composite electronic 
presentation with a browser interface for accessing a segment of video file which 
references scanned document- see Chui at Fig. 3 and Col. 4, Line 1-15.) 



In addition Chui and Hull do not teach, but Bozdaqi teaches: 

generating a user selectable object providing a user with 
access to the portion of the recorded information corresponding to 
the visual feature, and inserting the user selectable object into the 
electronic representation of the document when the computer 
system locates a portion of the recorded information determined to 
match the visual feature, including the user selectable object, and 
the user selectable object being placed in a position associated with 
the extracted feature and allowing the user to access the portion of 
the recorded information in an application displaying the ..electronic 
representation or a separate application by selecting the user 
selectable. 

(See Fig. 3-4 and the Abstract and Column 5, Line 25 through Column 6, Line 
65->Bozdagi disclosed generating a user selectable object providing a user with 
access to the portion of the recorded information corresponding to the visual 
feature, and inserting the user selectable object into the electronic representation 
of the document; as illustrated in Fig. 3, the selectable object items 1 15 are 
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generated and being inserted into the web document 210, wherein the text item 
1 17 is matching text for specific video link that is linkable to the video files [visual 
object] that allowed user to access the matching video as designed.) 
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FIG. 3 

Accordingly, It would have been obvious to one of ordinary skill in the art 
at the time of the invention to modify Chui and Hull method of linking a video to a 
scan document and comparing the visual feature to the recorded information to 
determine a portion of the recorded information corresponding to the visual 
feature, whereby at least a portion of the recorded information corresponds to a 
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feature portion of the presentation material to include a means of said generating 
a user selectable object providing a user with access to the portion of the 
recorded information corresponding to the visual feature, and inserting the user 
selectable object into the electronic representation of the document when the 
computer system locates a portion of the recorded information determined to 
match the visual feature, including the user selectable object being placed in a 
position associated with the extracted feature and allowing the user to access the 
portion of the recorded information in an application displaying the ...electronic 
representation or a separate application by selecting the user selectable as 
taught by Bozdagi. One of ordinary skill in the art would have been motivated to 
modify this combination because Chui, Hull and Bozdagi are from the same field 
of endeavor of electronic presentation material (i.e. multimedia) associated with 
the scan document, and provides a predictable result of generating a composite 
electronic presentation with a browser interface for accessing a segment of video 
file which references scanned document- see Chui at Fig. 3 and Col. 4, Line 1- 
15). 

Regarding Independent claim 21, 

is fully incorporated similar subject of claim 1 cited above, and is 
similarly rejected along the same rationale. Thus, Chiu, Hull and Bozdagi 
disclose every limitation of Claim 21 and provide proper reasons to 
combine, as indicated in the above rejection for Claim 1 . 
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Regarding Independent claim 34, 

Claim 34 recites a computer program product stored on a 
computer-readable medium for implement a method recited in Claim 1 . 
Thus, Chiu, Hull and Bozdagi disclose every limitation of Claim 34 and 
provide proper reason to combine, as indicated in the above rejections for 
Claim 1 - Also See Chiu at Fig. 1 . 

Regarding Independent claim 44, 

Claim 44 recites a computer program product stored on a 
computer-readable medium for implement a method recited in Claim 21 . 
Thus, Chiu, Hull and Bozdagi disclose every limitation of Claim 44 and 
provide proper reason to combine, as indicated in the above rejections for 
Claim 21 - Also See Chiu at Fig. 1 . 

Regarding Independent claim 51, 

Claim 51 recites a data processing system for implement a method 
recited in Claim 1. Thus, Chiu, Hull and Bozdagi disclose every limitation 
of Claim 51 and provide proper reason to combine, as indicated in the 
above rejections for Claim 1- Also See Chiu at Fig. 1 . 



Regarding Independent claim 61, 

Claim 61 recites a data processing system for implement a method 
recited in Claim 21 . Thus, Chiu, Hull and Bozdagi disclose every limitation 
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of Claim 61 and provide proper reason to combine, as indicated in the 
above rejections for Claim 61- Also See Chiu at Fig. 1 . 

Regarding Independent claim 68, 

Claim 68 recites a system for implement a method recited in Claim 
1 . Thus, Chiu, Hull and Bozdagi disclose every limitation of Claim 68 and 
provide proper reason to combine, as indicated in the above rejections for 
Claim 1 - Also See Chiu at Fig. 1 . 

Regarding Independent claim 69, 

Claim 69 recites a data processing system for implement a method 
recited in Claim 21 . Thus, Chiu, Hull and Bozdagi disclose every limitation 
of Claim 69 and provide proper reason to combine, as indicated in the 
above rejections for Claim 1- Also See Chiu at Fig. 1 . 

Regarding claims 2, 22, 35, 45, 52, and 62, 

Chiu, Hull and Bozdagi further comprise: 

determining associating information for the recorded 
information that corresponds to the extracted feature. 

(See Chiu at the Abstract, teaches a computer system for creating a composite 
electronic representation including presentation material information (i.e. 
annotations made on the paper documents during the meeting can be extracted 
and used as indexes to the video. Chiu further discloses a method for linking a 
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scanned document to a segment of a video is provided. Whereby the scanned 
document identifier is then compared to the plurality of video frame identifiers; 
the scanned document is linked to a first video frame in the plurality of video 
frames responsive to the comparison step-See Chiu at Col. 2, Lines 15-25, 
discloses. 

Also see Chiu at Fig. 2, illustrates bar 104a represents the time and 
duration in which document 101 is presented as projection 101b in window 203.) 

Regarding claims 3, 23, 46 and 63, 

Chiu, Hull and Bozdagi further comprise: 

wherein the association information comprises time 
information and source information for recorded information. 

(See Chiu at the Abstract, teaches a computer system for creating a composite 
electronic representation including presentation material information (i.e. 
annotations made on the paper documents during the meeting can be extracted 
and used as indexes to the video. Chiu further discloses a method for linking a 
scanned document to a segment of a video is provided. Whereby the scanned 
document identifier is then compared to the plurality of video frame identifiers; 
the scanned document is linked to a first video frame in the plurality of video 
frames responsive to the comparison step-See Chiu at Col. 2, Lines 15-25, 
discloses. 

Also see Chiu at Fig. 2, illustrates bar 104a represents the time and 
duration in which document 101 is presented as projection 101b in window 203.) 
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Regarding claims 4, 36, and 53, 

Chiu, Hull and Bozdagi further comprise: 

associating the association information with the determined 
additional information in the composite electronic representation. 

(See Chiu at the Abstract, teaches a computer system for creating a composite 
electronic representation including presentation material information (i.e. 
annotations (i.e. additional information) made on the paper documents during the 
meeting can be extracted and used as indexes to the video.) 

Regarding claims 5, 24, 37,47, 54 and 64, 

Chiu, Hull and Bozdagi further comprise: 

receiving a selection to the determined additional information 
in the composite electronic representation and using the association 
information for the additional information to access the recorded 
information. 

(See Chiu at Col. 2, Lines 15-25, discloses a method for linking a scanned 
document to a segment of a video is provided. Whereby the scanned document 
identifier is then compared to the plurality of video frame identifiers; the scanned 
document is linked to a first video frame in the plurality of video frames 
responsive to the comparison step. 

Also see Chiu at Fig. 3 and Col. 4, Line 1-15, discloses browser interface 
300 for accessing a segment of video file 200 which references scanned 
document 101a.) 
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Regarding claims 6, 25, 38 and 55, 

Chiu, Hull and Bozdagi further comprise: 

accessing the recorded information using the determined 
additional information. 

(See Chiu at Col. 2, Lines 15-25, discloses a method for linking a scanned 
document to a segment of a video is provided. Whereby the scanned document 
identifier is then compared to the plurality of video frame identifiers; the scanned 
document is linked to a first video frame in the plurality of video frames 
responsive to the comparison step. 

Also see Chiu at Fig. 3 and Col. 4, Line 1-15, discloses browser interface 
300 for accessing a segment of video file 200 which references scanned 
document 101a.) 

Regarding claims 7, and 26, 

Chiu, Hull and Bozdagi further comprise: 

displaying the accessed recorded information. 

(See Chiu at Fig. 3 and Col. 4, Line 1-15, discloses browser interface 300 for 
accessing a segment of video file 200 which references scanned document 
101a.) 

Regarding claims 8, and 27, 

Chiu, Hull and Bozdagi further comprise: 
playing the accessed information. 
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(See Chiu at Fig. 3 and Col. 4, Line 1-15, discloses browser interface 300 for 
accessing a segment of video file 200 which references scanned document 
101a.) 

Regarding claims 9, 29, 39, 49, 56 and 66, 

Chiu, Hull and Bozdagi further comprise: 

performing at least one of emailing, printing, storing, and 
copying the created composite electronic representation. 

(See Chiu at Fig. 5 and Col. 8 Line 5^discloses memory item 806 and persistent 
storage 808 stores a scanned digital document 101a and video file 200.) 

Regarding claims 11, 41 and 58, 

Chiu, Hull and Bozdagi further comprise: 

wherein the received electronic representation of the paper 
document includes notes taken by a user, wherein the created 
composite electronic representation includes the notes taken by the 
user. 

(See Chiu at Col. 7, Lines 1-10, discloses a method of extract the ink 
annotations, a simple comparison between the original and the annotated paper 
handout may be performed. When more than one set of handouts have been 
annotated, these ink strokes may be extracted and selectively layered over the 
common background of the scanned document. Another way to display the ink 
annotations and notes is simply to show them without a background. In any case, 
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the ink strokes may be hyperlinked to play their corresponding segment in the 
video recording.) 

Regarding claim 13, 

Chiu, Hull and Bozdagi further comprise: 

determining a document that includes the recorded 
information using the extracted feature. 

(See Chiu at Col. 7, Lines 1-10, discloses a method of extract the ink 
annotations, a simple comparison between the original and the annotated paper 
handout may be performed. When more than one set of handouts have been 
annotated, these ink strokes may be extracted and selectively layered over the 
common background of the scanned document. Another way to display the ink 
annotations and notes is simply to show them without a background. In any case, 
the ink strokes may be hyperlinked to play their corresponding segment in the 
video recording.) 

Regarding claim 14, 

Chiu, Hull and Bozdagi further comprise: 

determining a portion of the document that includes the 
information corresponding to the feature. 

(See Fig. 2-33 and Section 4.1 "Similarity detection" Pages 39-40^ Hull 
discloses this limitation in that features extracted from the CCITT fax- 
compressed representations 
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Of two images are compared to determine their visual similarity and whether they 
are equivalent.) 

Accordingly, it would have been obvious to one of ordinary skill in the art 
at the time of the invention to modify Chui and Bozdagi to include a means of 
determining a portion of the document that includes the information 
corresponding to the feature as taught by Hull. One of ordinary skill in the art 
would have been motivated to modify this combination because Hull and Chui 
are from the same field of endeavor of electronic presentation material 
associated with the scan document, and provides a predictable result of 
generating a composite electronic presentation with a browser interface for 
accessing a segment of video file which references scanned document- see Chui 
at Fig. 3 and Col. 4, Line 1-15.) 

Regarding claims 17-18, 

Chiu, Hull and Bozdagi further comprise: 

receiving the electronic representation comprises receiving a 
scan of the document, the document being a paper document, 
wherein receiving the electronic representation comprises 
determining an electronic image of the document, the document 
being a paper document. 
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(See Chiu at Col. 7, Lines 1-10, discloses a method of extract the ink 
annotations, a simple comparison between the original and the annotated paper 
handout may be performed.) 

Regarding claims 19, and 33, 

Chiu, Hull and Bozdagi further comprise: 

wherein receiving the electronic representation comprises 
receiving the electronic representation in response to an input from 
a user indicating that the composite electronic representation should 
be created. 

(See Chiu at the Abstract, teaches a computer system for creating a composite 
electronic representation including presentation material information (i.e. 
annotations made on the paper documents during the meeting can be extracted 
and used as indexes to the video. This interpretation is supported by Applicant's 
Specification, which states "creating a composite electronic representation 
comprising the determined information" at Pages 3 Para 15. ) 

Regarding claims 20, 30, 43, 60 and 67, 

Chiu, Hull and Bozdagi further comprise: 

the document comprises a paper document. 

(See Chiu at Col. 7, Lines 1-10, discloses a method of extract the ink 
annotations, a simple comparison between the original and the annotated paper 
handout may be performed.) 
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Claims 15-16, 31-32, 42, 50 and 59 are rejected under 35 U.S.C. 103(a) 
as being as being unpatentable over Chiu et al US007051271B1 filed 
10/03/2000 (hereinafter Chiu), in view of Hull [Document image similarity and 
equivalence protection] Published 11/29/119 by " IJDARUS" Pages 37-42 
(hereinafter Hull), and further view of Bozdaqi et al -US006647535B1 1 filed 
08/18/1999 (hereinafter Bozdagi), and further in view of Coar US 
200701 06932A1 Continuation of 09/533,152 filed 03/23/2000 (hereinafter Coar). 

Regarding claims 15, 31, 42, 50 and 59, 

Chui, Hull and Bozdagi do not expressly teach, but Coar teaches: 

an identifier to a location in the recorded information, wherein 

the information in the recorded information corresponding to the 

feature is determined using the identifier. 
(See Fig. 1 and at Para 86-> Coar discloses this limitation in that a user with 
access to the portion of the recorded information corresponding to the visual 
feature (i.e. XWPL (extensible Workflow Package Language is an extensible 
Markup Language (XML) based language. One portion of the language defines a 
standardized method to place information into a machine-readable symbol such 
as a high-density barcode) symbols (one symbol for each document and row in 
the data source) for each document to be scanned. The user would then prepare 
the documents with the XWPL symbol in the proper sequence to permit 
scanning, the user could then scan the documents, and the application would 
then, using the necessary symbol recognition tools, extract the data from the 
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symbol, and automatically place the image of the document within the correct 
VirPack in the correct placemen.) 

Accordingly, It would have been obvious to one of ordinary skill in the art 
at the time of the invention to modify Chui, Hull and Bozdagi to include a means 
of said an identifier to a location in the recorded information, wherein the 
information in the recorded information corresponding to the feature is 
determined using the identifier as taught by Coar. One of ordinary skill in the art 
would have been motivated to modify this combination because Chui, Hull, 
Bozdagi and Coar Hull are from the same field of endeavor of electronic 
presentation material associated with the scan document, and provides a 
predictable result of generating a composite electronic presentation with a 
browser interface for accessing a segment of video file which references 
scanned document- see Chui at Fig. 3 and Col. 4, Line 1-15.) 

Regarding claims 16 and 32, 

Chui, Hull and Bozdagi do not expressly teach, but Coar teaches: 
the identifier comprises at least one of a barcode and 
signature information. 

(See Coar at Fig. 1 and at Para 86, discloses a machine-readable symbol such 
as a high-density barcode.) 
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Accordingly, it would have been obvious to one of ordinary skill in the art 
at the time of the invention to modify Chui, Hull and Bozdagi to include a means 
of said the identifier comprises at least one of a barcode and signature 
information as taught by Coar. One of ordinary skill in the art would have been 
motivated to modify this combination because Chui, Hull, Bozdagi and Coar Hull 
are from the same field of endeavor of electronic presentation material 
associated with the scan document, and provides a predictable result of 
generating a composite electronic presentation with a browser interface for 
accessing a segment of video file which references scanned document- see Chui 
at Fig. 3 and Col. 4, Line 1-15.) 

It is noted that any citations to specific, pages, columns, lines, or figures in 
the prior art references and any interpretation of the references should not be 
considered to be limiting in any way. A reference is relevant for all it contains 
and may be relied upon for all that it would have reasonably suggested to one 
having ordinary skill in the art. See, MPEP 2123. 

Response to Arguments 

Brief description of cited prior art: 

Chiu at Fig. 2-3 and 5 disclosed browser interface (item 300) for 
accessing a segment of video file (item 200) which references scanned 
document (item 101a), wherein browser interface 300 includes window (item 
302) for viewing scanned documents (item 301 and 101a). This allows a user 
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may double-click on the scanned document (item 301 or 101a) and browser 
interface (item 100a) will play in window (item 203) the corresponding recorded 
segment of event (item 50) referencing the scanned documents. Also the 
scanned documents (item 301 and 101a) may be represented by hyperlinked 
universal resource location ("URL") addresses, where the video may be resided- 
See Chiu at Fig. 1-5 and at Column 3, Line 45 through Column 4, Line 15. Also 
Chiu further disclosed the links between a scanned document and a segment of 
video matching the scanned document (col. 2, lines 15-17; abstract). Coefficients 
of an image of the document and coefficients of representative video frames are 
compared, and when the coefficients match within a predetermined threshold 
the document is linked or indexed to the video frame (col. 6, lines 5-45). Also 
Chiu further illustrated at FIG. 2 browser interface 100a having window 105 
displays the relative time in which video frame 200a is displayed during the 
course of event 50 [See Chiu at Fig. 2 Col. 2, Lines 15-25]; This interpretation is 
supported by Applicant's Specification, which states "metadata may indicate how 
long the slide was discussed" at Pages 14 Para [0066] Lines 11-12.) 

Hull et al. disclosed extracted from the CCITT fax-compressed 
representations and compress images form database are compared to determine 
their visual similarity and whether they are equivalent. As illustrated in Fig. 2 
hierarchical algorithm is presented for determining the similarity and equivalence 
of document images which are extracted from the CCITT fax-compressed 
representations of two images are compared to determine their visual similarity 
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and whether they are equivalent. The equivalence of two documents is 
determined by applying the Hausdorff distance to the two-dimensional 
arrangement of pass codes in small patches of each image [matching portion of 
recorded images]. In this case the fax-compress images is compared to the 
compress image form the data base to determine their visual similarity and 
whether they are equivalent [See Fig. 2 and at Pages 308-31]. Furthermore, Hull 
further disclosed the similarity detection algorithm include incorporation of 
structural information about the segmentation of a document into zones such 
as text, graphics, photographs, etc. Such information may improve 
performance. However, the run time overhead of such a method should be 
considered. The experiments with the equivalence detection method assume 
corresponding image patches can be located in two equivalent images. An 
automatic technique for calculating this from compressed data should be 
developed [at Page 312 Top right column of Hull.) 

Bozdaqi et al. disclosed the method and system for generating and 
inserting a user selectable object providing a user with access to the portion of 
the recorded information corresponding to the visual feature of the electronic 
representation of the document; as illustrated in Fig. 3, the selectable object 
items 1 15 are generated and being inserted into the web document 210, wherein 
the text item 1 17 is matching text for specific video link that is linkable to the 
video files [visual object] that allowed user to access the matching video as 
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designed [See Fig. 3-4 and the Abstract and Column 5, Line 25 through Column 
6, Line 65 of Bozdagi] 
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Coar disclosed a machine readable symbol at the time which the paper 
document printed, wherein those machine readable symbol are programmatic 
interface to permit interaction with all portions of the package (i.e. the 
resulting electronic package file would then contain an image of the paper 
document and the index data in a form that permits easy extraction and transfer 
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to another system- See Coar at Para 31), permit security access, edit, view, 
extraction, processing, and delivery of contents- See Coar at Para 23, 31 , and 
41-42- [see above rejection of claims 16 and 32 for further details]. 

Response to Remarks: 

Beginning on pages 18/22 of the Remarks filed 10/16/2008 (hereinafter 
the remarks), Applicants argue the following issues, which are accordingly 
addressed below. 

Regarding rejections of claims 1-9; 11; 13-27; 29-39; 41-47; 49-56; 58-64 and 
66-69 filed 10/16/2008 under 35 USC. 103: 

Applicant asserts that the proposed combination [of Chiu; Hull; Bozdagi 
and Coar] fails to established a prima facie case, because Chiu and Hul and 
Bozdagi fail tto teach, "comparing the visual feature to the recorded 
information.. .; and genenerating...and inserting a user selectable obejct provide 
user with access to the portion of the record information..." [see te remarks 
pages 19, line 8 through Page 20 line 23. 

For purposes of responding to Applicant's argument, the examiner will 
assume that Applicant is arguing for the patentability of independent claims 1 , 21 , 
34,44, 51,61,68, and 69. 
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The Examiner disagrees. 

As discuss above, and recognized by the Examiner, Chiu at Fig. 2-3 and 
5 disclosed browser interface (item 300) for accessing a segment of video file 
(item 200) which references scanned document (item 101a), wherein browser 
interface 300 includes window (item 302) for viewing scanned documents (item 
301 and 101a). This allows a user may double-click on the scanned document 
(item 301 or 101a) and browser interface (item 100a) will play in window (item 
203) the corresponding recorded segment of event (item 50) referencing the 
scanned documents. Also the scanned documents (item 301 and 101a) may be 
represented by hyperlinked universal resource location ("URL") addresses, 
where the video may be resided- See Chiu at Fig. 1-5 and at Column 3, Line 45 
through Column 4, Line 15. Also Chiu further disclosed the links between a 
scanned document and a segment of video matching the scanned document 
(col. 2, lines 15-17; abstract). Coefficients of an image of the document and 
coefficients of representative video frames are compared, and when the 
coefficients match within a predetermined threshold the document is linked or 
indexed to the video frame (col. 6, lines 5-45). Also Chiu further illustrated at 
FIG. 2 browser interface 100a having window 105 displays the relative time in 
which video frame 200a is displayed during the course of event 50 [See Chiu at 
Fig. 2 Col. 2, Lines 15-25]; 

This interpretation is supported by Applicant's Specification, which states 
"metadata may indicate how long the slide was discussed" at Pages 1 4 Para 
[0066] Lines 11-12. 
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In addition, "What matters is the objective reach of the claim. If the claim 
extends to what is obvious, it is invalid under § 103." KSR Int'l Co. v. Teleflex, 
Inc., 127 S. Ct. 1727, 1742 (2007). To be nonobvious, an improvement must be 
"more than the predictable use of prior art elements according to their 
established functions." Id. at 1740. 

As recognized by the Examiner, Chiu's browser interface for accessing a 
segment of video file which references scanned document whereby Video 
recordings of meetings and scanned paper documents are natural digital 
documents that come out of a meeting [see Chiu at the title and the abstract]. As 
recognized by the Examiner, Chiu does not expressly teach the use of comparing 
the visual feature to the recorded information as recited in independent claims 1 , 
21 , 34, 44, 51 , 61 , 68, and 69. On the other hand, in what is fairly characterized 
as analogous art in accordance with the above-noted case law, Hull et al. 
disclosed extracted from the CCITT fax-compressed representations and 
compress images form database are compared to determine their visual 
similarity and whether they are equivalent. As illustrated in Fig. 2 hierarchical 
algorithm is presented for determining the similarity and equivalence of document 
images which are extracted from the CCITT fax-compressed representations of 
two images are compared to determine their visual similarity and whether 
they are equivalent. The equivalence of two documents is determined by 
applying the Hausdorff distance to the two-dimensional arrangement of pass 
codes in small patches of each image [matching portion of recorded images]. In 
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this case the fax-compress images is compared to the compress image form the 
data base to determine their visual similarity and whether they are equivalent 
[See Fig. 2 and at Pages 308-31]. Furthermore, Hull further disclosed the 
similarity detection algorithm include incorporation of structural information about 
the segmentation of a document into zones such as text, graphics, 
photographs, etc. Such information may improve performance. However, the 
run time overhead of such a method should be considered. The experiments with 
the equivalence detection method assume corresponding image patches can be 
located in two equivalent images. An automatic technique for calculating this 
from compressed data should be developed [at Page 312 Top right column of 
Hull. Accordingly, It would have been obvious to one of ordinary skill in the art at 
the time of the invention to modify Chui method of linking a video to a scan 
document to include a means of comparing the visual feature to the recorded 
information to determine a portion of the recorded information that matches the 
visual feature, whereby at least a portion of the recorded information matches a 
feature portion of the presentation material, and determining matching 
information for each matching portion of the recorded information and feature 
portion using a matching algorithm configured to map the visual feature to a 
portion of any of a plurality of recorded information that matches the feature 
portion as taught by Hull. One of ordinary skill in the art would have been 
motivated to modify this combination because Hull and Chui are from the same 
field of endeavor of electronic presentation material associated with the scan 
document, and provides a predictable result of generating a composite electronic 
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presentation with a browser interface for accessing a segment of video file which 
references scanned document- see Chui at Fig. 3 and Col. 4, Line 1-15.) 

In additions, as recognized by the Examiner, Chiu's browser interface for 
accessing a segment of video file which references scanned document 
whereby Video recordings of meetings and scanned paper documents are 
natural digital documents that come out of a meeting [see Chiu at the title and the 
abstract] and Hull's compared to determine their visual similarity and 
whether they are equivalent [See Fig. 2 and at Pages 308-31], in combination 
do not expressly teach the use of genenerating...and inserting a user selectable 
obejct provide user with access to the portion of the record information as recited 
in independent claims 1,21, 34, 44, 51 , 61 , 68, and 69. On the other hand, in 
what is fairly characterized as analogous art in accordance with the above-noted 
case law, Bozdaqi et al. disclosed the method and system for generating and 
inserting a user selectable object providing a user with access to the portion of 
the recorded information corresponding to the visual feature of the electronic 
representation of the document; as illustrated in Fig. 3, the selectable object 
items 1 15 are generated and being inserted into the web document 210, wherein 
the text item 1 17 is matching text for specific video link that is linkable to the 
video files [visual object] that allowed user to access the matching video as 
designed [See Fig. 3-4 and the Abstract and Column 5, Line 25 through Column 
6, Line 65 of Bozdagi]. Accordingly, It would have been obvious to one of 
ordinary skill in the art at the time of the invention to modify Chui and Hull method 
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of linking a video to a scan document and comparing the visual feature to the 
recorded information to determine a portion of the recorded information 
corresponding to the visual feature, whereby at least a portion of the recorded 
information corresponds to a feature portion of the presentation material to 
include a means of said generating a user selectable object providing a user with 
access to the portion of the recorded information corresponding to the visual 
feature, and inserting the user selectable object into the electronic representation 
of the document when the computer system locates a portion of the recorded 
information determined to match the visual feature, including the user selectable 
object as taught by Bozdagi. One of ordinary skill in the art would have been 
motivated to modify this combination because Chui, Hull and Bozdagi are from 
the same field of endeavor of electronic presentation material (i.e. multimedia) 
associated with the scan document, and provides a predictable result of 
generating a composite electronic presentation with a browser interface for 
accessing a segment of video file which references scanned document- see Chui 
at Fig. 3 and Col. 4, Line 1-15). 

Thus Chiu; Hull and Bozdagi clearly disclosed the all the limtaiton of 1 , 21 , 
34, 44, 51 , 61 , 68, and 69; particulally comparing the visual feature to the 
recorded information...; and genenerating...and inserting a user selectable obejct 
provide user with access to the portion of the record information..., as recited in 
1 , 21 , 34, 44, 51 , 61 , 68, and 69, and provided proper reasons to combine [see 
also the above rejections for details] 
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Regarding rejections of claims 15-16. 31-32, 42, 50, and 59 under 35 USC. 103: 

Applicant asserts that the proposed combination [of Chiu; Hull; Bozdagi 
and Coar] fails to established a prima facie case, because Coar fails to teach, 
"comparing the visual feature to the recorded information...; and 
genenerating...and inserting a user selectable obejct provide user with access to 
the portion of the record information..." [see the remarks page 21 , Lines 15-28] 
and because Coar "can not have a machine readable symbol included at the 
tiem of printing" see the remarks at Page 21 Lines 6-9. 

The Examiner disagrees. 

As discuss above, Chiu; Hull and Bozdagi clearly disclosed the all the 
limtaiton of 1 , 21 , 34, 44, 51 , 61 , 68, and 69; particulally comparing the visual 
feature to the recorded information...; and genenerating...and inserting a user 
selectable obejct provide user with access to the portion of the record 
information..., as recited in 1, 21, 34, 44, 51, 61, 68, and 69, and provided proper 
reasons to combine [see also the above rejections and responses to the remarks 
to claims , 21 , 34, 44, 51 , 61 , 68, and 69 cites above for details] 

In addition, The Examine believes the applicant arguments; specifically 
the limitation of "can not have a machine readable symbol included at the tiem of 
printing" is not positively recited in the claim language. For further clarification, as 
explained in the above rejection for Claims 15-16, 31-32, 42, 50, and 59, Coar 
disclosed a machine readable symbol at the time which the paper document 
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printed, wherein those machine readable symbol are programmatic interface to 
permit interaction with all portions of the package (i.e. the resulting electronic 
package file would then contain an image of the paper document and the index 
data in a form that permits easy extraction and transfer to another system- See 
Coar at Para 31), permit security access, edit, view, extraction, processing, and 
delivery of contents- See Coar at Para 23, 31 , and 41-42- [see above rejection of 
claims 16 and 32 for further details]. Also Coar further disclosed a method for 
creating and using virtual electronic containers for documents and data. The 
electronic container provides a capability to scan and store paper documents in a 
virtual container. The container can also store existing electronic documents with 
associated data. Electronic documents are stored and maintained in their original 
format, allowing extraction and use by the applications that created them. 
The electronic container also recognizes graphical symbols of prescribed 
formats. This is generally disclosed at the Abstract of Coar. Accordingly, it would 
have been obvious to one of ordinary skill in the art at the time of the invention to 
modify Chui, Hull and Bozdagi to include a means of said the identifier comprises 
at least one of the barcode and signature information as taught by Coar. One of 
ordinary skill in the art would have been motivated to modify this combination 
because Chui, Hull, Bozdagi and Coar Hull are from the same field of endeavor 
of electronic presentation material associated with the scan document, and 
provides a predictable result of generating a composite electronic presentation 
with a browser interface for accessing a segment of video file which references 
scanned document- see Chui at Fig. 3 and Col. 4, Line 1-15.). 
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Thus Chiu; Hull and Bozdagi and Coar clearly disclosed the all the 
limtaiton of 15-16, 31-32, 42, 50, and 59; particulally comparing the visual feature 
to the recorded information...; and genenerating...and inserting a user selectable 
obejct provide user with access to the portion of the record information..., 
wherein an identifier to a location in the recorded information, wherein the 
information in the recorded information corresponding to the feature is 
determined using the identifier [e.g. barcode] as recited in 15-16, 31-32, 42, 50, 
and 59, and provided proper reasons to combine [see also the above rejections 
for details] 

Therefore claims 1-9; 1 1 ; 13-27; 29-39; 41-47; 49-56; 58-64 and 66-69 
remains rejected at least at this time. 

Conclusion 

THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of 
time policy as set forth in 37 CFR 1 .1 36(a). 

A shortened statutory period for reply to this final action is set to expire 
THREE MONTHS from the mailing date of this action. In the event a first reply is 
filed within TWO MONTHS of the mailing date of this final action and the advisory 
action is not mailed until after the end of the THREE-MONTH shortened statutory 
period, then the shortened statutory period will expire on the date the advisory 
action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be 
calculated from the mailing date of the advisory action. In no event, however, will 
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the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 

Any inquiry concerning this communication or earlier communications from 
the examiner should be directed to Quoc A. Tran whose telephone number is 
571-272-8664. The examiner can normally be reached on Monday through 
Friday from 9 AM to 5 PM EST. 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Doug Hutton can be reached on 571-272-4137. The fax 
phone number for the organization where this application or proceeding is 
assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from 
the Patent Application Information Retrieval (PAIR) system. Status information 
for published applications may be obtained from either Private PAIR or Public 
PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see http://pair- 
direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll- 
free). 

/Quoc A. Tran/ 
Patent Examiner 



/DOUG HUTTON/ 

Supervisory Patent Examiner, Art Unit 2176 



